Semantic Mapping with Human-Robot Interaction

Knowledge representation in robotics is a challenging problem. Both the chosen abstraction and the type of information, in fact, may affect the performance of the robot. In particular, a key problem to take into account is symbol grounding, since the symbols used in the representation need to be connected to the proper elements of the operational environment. In the case of robots that operate in a physical environment, symbol grounding requires to “close the loop” with perception, thus defining the relationship between numeric and symbolic representation. State-of-the-art systems, typically support the implementation of simple commands, through a significant engineering effort in coding the knowledge about the specific operational environment and implementing the required perception components. However, such an approach does not scale, since the robot cannot be used in different environments and settings. Furthermore, hard-coding this type of knowledge produces a representation which cannot be re-used by other devices. On the contrary, dealing with many objects and environments, and being aware of spatial relations between objects and functional areas, are important features for service robots. For these reasons, many researchers tried to represent knowledge about the environments through the use of semantic maps. The resulting abstraction is typically a map that integrates the metric information used for navigation with labels providing a semantic characterization of places and objects. Even if semantic maps represent a significant step forward in the direction of environmental knowledge representation, they currently suffer from the limitations related to the use of perception data; moreover, current methods for semantic mapping are typically designed to work offline, focusing on the process that enables the robot to build a suitable representation, before operating in a given environment. On the contrary, this work aims at enabling robots to add semantic knowledge to the map, by a continuous lifelong interaction process. To this end, our approach is based on human-robot collaboration, which allows non-expert operators to generate rich semantic information about an environment. This is obtained through a multi-modal natural interaction framework that supports the semantic map building process (see Fig. 1). Even if automatic methods are still important in the definition of an effective semantic map, collaboratively acquiring knowledge through human-robot interaction can increase the overall performance of the system. Since the robot requires the help of the user whenever it cannot ground a command in the map, any uncertainty or inconsistency in symbol grounding or in object Fig. 1. Left: interactive tagging process in a domestic environment; Right: semantic map generated during the interaction